Commentary on "Near-Optimal Algorithms for Online Matrix Prediction"

This piece is a commentary on the paper by Hazan et al. (2012b). In their paper, they introduce the class of (β, τ)-decomposable matrices, and show that well-known matrix regularizers and matrix classes (e.g. matrices with bounded trace norm) can be viewed as special cases of their construction. The β and τ terms can be related to the max norm and to the trace norm, respectively, as explored in the paper, which we discuss in detail below. The paper’s main contribution is a powerful online learning guarantee when learning inside the (β, τ)-decomposable class, which scales with √ β · τ , and an efficient algorithm for solving this learning problem. Crucially, the paper reframes the well-known problems of online max cut, learning a team ranking (“gambling”), and trace-norm regularized matrix completion (a.k.a. collaborative filtering) as special cases of learning inside (β, τ)-decomposable classes of matrices. This yields new algorithms for the three existing problems, with each algorithm giving a strong improvement over existing results in terms of either efficiency or error rate guarantees. In addition, the paper derives lower bounds on the error rates for each of the three problems that match (up to log factors) the upper bounds proved with the new algorithm—and in particular, for collaborative filtering with the trace norm, their lower bound solves an open problem posed by Shamir and Srebro in COLT 2011. In this commentary, we explore the connections between the class of (β, τ)-decomposable matrices, introduced by Hazan et al. (2012b), and the matrix trace norm (a.k.a. nuclear norm) and max norm. Specifically, we are interested in the idea of a “trade-off” between the β and τ values for the class, and will consider how the resulting non-convex optimization question can be formulated as a series of convex optimization problems.